Process for the indentification of objects

ABSTRACT

The invention is a process for identifying an unknown object. In detail, the process includes the steps of: 1) compiling data on selected features on a plurality of segments of a plurality of known objects; 2) illuminating the unknown object with a laser radar system; 3) dividing the unknown object into a plurality segments corresponding to each of the segments of the known objects; 4) sequentially measuring selected features of each of the plurality of segments of the unknown object; and 5) comparing the sequentially measuring selected features of each of the plurality of segments of the unknown object to the selected features on the plurality of segments of the plurality of known objects.

GOVERNMENT INTEREST

This invention was made under US Government Contract No.:FZ6830-01-D-002 issued by the US Air Force dated March 2004. Therefore,the US Government has the rights to the invention granted thereunder.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to the field of identification of objects such asstructures and vehicles and, in particular, to a process for theidentification of structures and vehicles using laser radars.

2. Description of Related Art

The identification of targets under battlefield conditions is a majorproblem. Of course, direct visual contact by trained personnel is themost accurate, but this exposes them to possible attack andsignificantly increases personnel workload. Thus in recent years, theuse of unmanned surveillance vehicles, particularly unmanned aircraft,have been used for battlefield surveillance. However, to avoid constantmonitoring of the unmanned vehicle; they are being equipped withautonomous systems that identify and classify potential targets and onlyinform the remotely located operator when such a target is identified.

Traditional Laser radar identification techniques have limitations inidentification of articulated targets because of the large number ofpotential target states due to the large number of potential targetarticulation, variations, and pose. The utility of invariant featuresfor the model-based matching of the entire target will reduce the searchspace but will not yield reliable estimates of target identification andpose. One approach is use a laser radar system to map the vehicle andrecord invariant parameters. These observed parameters are compared tothose stored in a data base to find a match. However, this method hasproved to be cumbersome to implement; because the whole structure,typically vehicles such as tanks or missile launchers, had to becompared to every other structure in the data base.

Thus, it is a primary object of the invention to provide a process forthe identification of objects without human intervention.

It is another primary object of the invention to provide a process forthe identification of unknown objects without human intervention thatuses a laser radar for illumination.

It is a further object of the invention to provide a process for theidentification of objects without human intervention that uses a laserradar for illumination and which provides optimum identification withminimum computing time.

SUMMARY OF THE INVENTION

The invention is a process for identifying an unknown object. In detail,the process includes the steps of:

1. Compiling data on selected features on a plurality of segments of aplurality of known structures. Preferably, the plurality of segmentsincludes the top and bottom or the top, middle and bottom of the object.This also includes the step of making piecewise pixel-pair invariantmeasurements of each of the plurality of segments of the knownstructures.

2. Illuminating the unknown structure with a laser radar system;

3. Dividing the unknown structure into a plurality segmentscorresponding to each of the segments of the known structures;

4. Sequentially measuring selected features of each of the plurality ofsegments of the unknown structure. This includes the steps of makingpiecewise pixel-pair invariant measurements of each of the plurality ofsegments of the unknown structure; and comparing the piecewisepixel-pair invariant measurements of each of the plurality of segmentsof the unknown structure until a match is found. This includes thedistance between first and second pixels of the pixel-pairs, the anglebetween the normals to the surface area about the first and secondpixels, the normalized distance between the first and second pairsprojected along the vector that is the sum of the two normals, and thenormalized distance between the first and second pixels along the vectorthat is the cross product of the two normals.

The novel features which are believed to be characteristic of theinvention, both as to its organization and method of operation, togetherwith further objects and advantages thereof, will be better understoodfrom the following description in connection with the accompanyingdrawings in which the presently preferred embodiment of the invention isillustrated by way of example. It is to be expressly understood,however, that the drawings are for purposes of illustration anddescription only and are not intended as a definition of the limits ofthe invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of the process to establish a reference library.

FIG. 2 is a perspective view of a tank

FIG. 3 is a perspective view of a missile launcher vehicle

FIG. 4 is a perspective view of a missile launcher illustrating theposition of a laser radar during the scanning of the launcher.

FIG. 5 is representation of a patch around a pixel; of a pixel pairillustrating the calculation of the normal vector.

FIG. 6 is representation of the neural used to process the pixel pairinvariants.

FIG. 7 is perspective view of a missile launcher and an aircraftillustrating the determination of the normal to the surface upon whichthe launcher rests.

FIG. 8 is a flow chart of the process to identify an unknown object

FIG. 9A, 9B, 9C, and 9D are four parts to a test results summary.

DESCRIPTION OF THE PREFERRED EMBODIMENT

The invention is a process for identifying potential targets by means ofa laser radar system that does not require human involvement. It isdesigned for use on an unmanned surveillance vehicle. This process isused after the vehicle has determined that a potential target exists.

In surveillance mission, the unmanned vehicle is sent to a target areabased on cues from intelligence gathered or cues from other long-rangesurveillance platforms. Due to target location error and potentialtarget movements, the unmanned vehicle needs perform its own search uponarrival to the target area using wide footprint sensors such asSynthetic Aperture Radar (SAR) or wide field of view Infrared Sensor.Upon detection of potential regions of interest (ROIs), the Laser Radarsensor is then cued to these ROIs to re-acquire and identify the targetand select the appropriate aim point to enhance weapon effectiveness,and reduce fratricide due to enemy fire.

Referring to FIG. 1, in detail the invention involves the followingsteps:

Step 10—Set Up A library Of Target Descriptions

Step 10A—Divide Objects in Segments that can be articulated. Allstructures that are of interest are first scanned by a laser radarsystem using simulation or actual data collection. The object is dividedinto sections. For example, referring to FIGS. 2 and 3, a tank 12 wouldbe divided into a hull 12A, turret 12B and gun 12C, a SA-6 missilelaunching vehicle 14 would be divided into a hull 14A, missile carriage14B and missiles 14C.

Step 10B—Scan Object Segment. Thereafter each section is scanned by thelaser radar system at various positions in a spherical pattern atapproximately two degree steps are made as illustrated in FIG. 4. Thiscan be accomplished using high fidelity physics-based modeling tools togenerate target signature using laser radar in simulation or usingactual sensor in a field data collection.Step 10C—Compute Angle Between Pixel Pairs: The angle between every twopairs of normals is computed using the dot product of the two normals${\cos\quad\theta} = \frac{n_{I}*n_{J}}{{n_{I}}*{n_{J}}}$Where:n_(I)=the unit vector representing the surface normal at one pixeln_(J)=the unit vector representing the surface normal at one pixel.=refers to the dot product between two vectorsOnly those pixel pairs having angles between 80 and 100 degrees aresaved.

The process continues with the following steps:

Step 10D-Calculate Invariants: After each measurement four invariantfeatures are recorded for each pixel pair. Referring to FIGS. 2 and 5these includes:

1. The distance between the first (P) and second (Q) pixels of thepixel-pairs,A _(INV) =∥P−Q∥  (1)Where: A_(inv)=Distance between pixel points2. The angle between the normals, {circumflex over (N)}_(P) and{circumflex over (N)}_(Q), to the to the surface area about the firstand second pixels,B _(INV)=cos⁻¹({circumflex over (N)} _(P) *{circumflex over (N)}_(Q))  (2)Where:{circumflex over (N)}_(P)=the normalized normal vector to the surface atpixel p computed using neighboring pixels{circumflex over (N)}_(Q)=the normalized normal vector to the surface atpixel Q computed using neighboring pixelsB_(INV)=Angle between {circumflex over (N)}_(P) and {circumflex over(N)}_(Q)3. The normalized distance between the first and second pairs along{circumflex over (N)}_(P)+{circumflex over (N)}_(Q),$C_{INV} = {{\left( {P - Q} \right)_{N}*\frac{\left( {P - Q} \right)}{\left( {{\hat{N}}_{P} + {\hat{N}}_{Q}} \right)/2}}}$Where:C_(inv)=normalized distance {circumflex over (N)}_(P)+{circumflex over(N)}_(Q),4. The normalized distance between the first and second pairs along{circumflex over (N)}_(P)*{circumflex over (N)}_(Q).D _(inv)=(P−Q)_(N)*(N _(P) *N _(Q))Where:D_(INV)=normalized distance between along the cross product of the twonormals {circumflex over (N)}_(P)×{circumflex over (N)}_(Q)The normal {circumflex over (N)}_(P) is determined by the use of theleased squared error method as illustrated in FIG. 5. $\begin{matrix}{{B\left( {i,j} \right)} = {{\sum\limits_{K = 1}^{9}{Pixel}_{K}} - {Target\_ Center}}} & (1) \\{{A\left( {i,j} \right)} = {\sum\limits_{K = 1}^{9}{{\begin{matrix}X_{K} \\Y_{K} \\Z_{K}\end{matrix}}*\left( {X_{K}Y_{K}Z_{K}} \right)}}} & (2) \\{{X\left( {i,j} \right)} = {{A\left( {i,j} \right)}^{- 1}*{B\left( {i,j} \right)}}} & (3) \\{{n_{P}^{\prime}\left( {i,j} \right)} = {{{Patch}\quad{{normall}\left( {i,j} \right)}} = \frac{X\left( {i,j} \right)}{{X\left( {i,j} \right)}}}} & (4) \\{{{Patch}\quad{Centroid}\quad\left( {i,j} \right)} = \frac{B\left( {i,j} \right)}{9}} & (5)\end{matrix}$  Patch Offset(I,j)=Patch Normal(I,j)*PatchCentroid(I,j)  (6)n _(P)(i,j)=one of the predefined normals closest to Nn′ _(P)  (7)Where X_(K)Y_(K)Z_(K) are the Cartesian coordinates of pixel (I,j)

Thus once the normal vectors for the two points (P, Q) are determined,the other invariant features are determined. Only those invariants thathave normal vectors between two points between 80 and 100 degrees areused. This will significantly reduce computational complexity and reducethe classification ambiguities due to excessive data.

The process continues with the following steps:

Step 10E—Prepare Multi-Dimensional Histograms. The four invariantsmeasured are first normalized by dividing the largest value for thatinvariant into all the other values thereof creating numbers varyingfrom zero to one. There a four dimensional array of 81 bins (3×3×3×3=81)is constructed. That is three bins for each variant:Bin Size A _(INV)=(max(A _(INV))−min(A _(INV)))/3)Bin Size B _(INV)=(max(B _(INV))−min(B _(INV)))/3Bin Size C _(INV)=(max(C _(INV))−min(C _(INV)))/3Bin Size D _(INV)=(max(D _(INV))−min(D _(INV)))/3A bin is determined for each pair of pixels:Index A _(INV)=(INT)((A _(INV))−min(A _(INV)))/BinSizeA _(INV)Index B _(INV)=(INT)((B _(INV))−min(B _(INV)))/BinSizeB _(INV)Index C _(INV)=(INT)((C _(INV))−min(C _(INV)))/BinSizeC _(INV)Index D _(INV)=(INT)((D _(INV))−min(D _(INV)))/BinSizeD _(INV)The number of bins is somewhat arbitrary, but testing has shown thatexcellent results are obtained using only 3 bins.Step 10F—Determine If All Segments Measured. If yes to Step 10G, if noto Step 10HStep 10G—Go to Next Segment—The program returns to Step 10BStep 10H—Determine if All Aspects Covered. In this step, a determinationis made as to whether all aspects have been covered by taking readingsat two degree increments around and over the object as illustrated inFIG. 4. If all aspects have not been covered then go to Step 10I; ifyes, then go to step 10JStep 10I—Go to next aspect—The laser radar is repositioned and theprogram returns to Step 10BStep 10J—Determine If There Is Another object. A determination as towhether another object is to be added to program, if yes to Step 10K, ifno to step 10L.Step 10K, a new object is selected and the program returns to Step 10A;

The process continues with the following steps:

Step 10L—Create Analysis Tool. The data created during Step 10E is fedto the neural net shown in FIG. 6. The net includes 81 input neurons. Ithas input layer, a hidden layer, and an output layer. The output layerhas a number of nerons equal to the total number of sections of alltargets that the neural network is trained on. The neural net is trainedto provide a value of one for a single output neuron with all the othersequal to zero. Thus for the tank hull 12A shown in FIG. 1 the outputneuron 24A would have a value of 1 and all the remaining neurons wouldbe equal to zero. Thereafter the data from the histogram would be fedinto the neural net to train it. For example, for the turret 12B, thevalue of the second output neuron 24B would be set to 1 and all othersset at zero and so. This would be repeated for every segment of everyobject to be placed in the library. A decision tree or a Hash tablecould be substituted for the neural net analysis tool.

Referring to FIG. 7, the library is loaded into a computer on anaircraft 28 having a laser radar 30. When an object of interest, forexample the tank 12 shown in FIG. 2 (but unknown to the computer onbroad the aircraft 28), the computer will analyze the object in thefollowing manner. Preferably, the aircraft 28 should be at a 30 to 60degree depression angle to the unknown object.

Step 34—Determine Normal To Ground Patch Around Vehicle—Still referringto FIG. 7, the ground segment 40 around the tank is at an angleindicated by numeral 42, is input to the program. Note that the aircraft28 is located at an altitude indicated by numeral 44 and having a GPSsystem knows its position above the ground 46. An arbitrary set of fourpoints 48A, 48B, 48C and 48D about the tank 12 are used to define asurface patch 50. The position of the four points 48A-48D can becomputed based upon the four laser beams 52A, 52B, 52C and 52D traveltime to the points and return and at the angles 54A, 54B, 54C and 54Dcan be used to computer their distance from the aircraft 28. The groundsegment is used to compute the normal vector to the ground plane asgiven by the following equations for which the normal 28 of the groundsegment 11A can be computed.

“Where are the equations?” Step 58—Rotate Object. The tank ismathematically rotated by use of the following equation:x′=x*cos(β)+y*sin(α)*sin(β)+z*cos(α)*sin(β)y′=y*cos(α)−z*sin(α)z′=y*sin(α)*cos(β)+z*cos(α)*cos(β)−x*sin(β)where x, y, z represent the original x, y, z coordinates and x′, y′, z′represent the newly rotated coordinatesα=the rotation angle about the x axis,β=the rotation angle about the y axis.Step 62-Select an object from the library, for example, the tank 12(FIG. 2) which is divided into three segments.Step 64—Set Height Boundary. The height of the target segment is setalong the normal. The bottom segment is first selected. Step 66—Makepixel Measurements. Using the laser radar pixel measurements are madeduring the laser radar scanning of the object, only one snap shot isrequired.Step 68—Compare angles between Normals—The normal to the surfaces aroundevery detected pixel is estimated using the procedure previouslydiscussed in setting up the library. Only pixel pairs whose anglebetween the normals have values within 80 and 100 degrees are consideredfor the classification process.

The process continues with the following steps:

Step 70—Computer Invariant Features.

Step 72—Prepare Multi-Dimensional Histograms.

Step 74—Analize Data. Using the trained neural net shown in FIG. 6.

Step 76—All Segments Examined?. If all segments examined go to Step 80,if not return to Step 64 and repeat process for next segment until allsegments of the object has been analyzed.

Step 78—All Known Objects Examined?. If no return to Step 62 and repeatprocess for next stored object data in reference library. If yes to Step80.

Step 80—Determine Unknown Object. The Score from the neural net can varyfrom 0 to 1; however, a score above 0.90, with all other scores below0.20 can be considered a positive identification. Several Events canoccur.

1. The object is identified

2. No identification is made

3. A multi-number of possible identities are produced.

In the first case, no further processing is required. In the second andthird case, the process can be terminated with a conclusion of “noidentification possible.” Some targets are very similar (such BTR-60,BTR70, and BTR-80 trucks). The classifier based on the laser radarresolution may not be capable to detect adequate details to discriminateamong these target types. Therefore if the classification belongs to anambiguous class, then there will be a request for refined sensorresolution. A sensor modality change is requested to change resolutionand re-image and re-segment the target area again. The segmented targetis then fed to the software to resolve ambiguity among the similartargets. The corresponding model of the target with the computedarticulation state is used to render the model using the sensorparameter file and Irma system (Government multi-sensor simulation thatis used to simulate target signatures from target models and sensorparameters). The simulated target signature is compared with the sensedtarget for final validation.

Additionally, the process starting at Step 64 can be repeated startingwith the top segment and working downward to achieve a higher confidencelevel. It may also establish the identification of a target object whenthe process measures from the top down. This is because the bottomsegment may be obscured by mud or foliage.

The process can also be used to assess battle damage or variation to agiven target segment by computing a transformation, which consists ofrotation and translation to compare the target piece to the model piece.The transformation equation isY=AX+b  (9)Where:A=rotation matrixb=translation vectorX and Y=are pixels on the target piece and the model piece.The above transformation is applied to the target piece to line up withthe corresponding model piece. The target piece is then subtracted fromthe model piece and the residual represents variation or battle damage.The residual can be used to infer the size of variation or the battledamage. All the classification hypotheses corresponding to the varioussegments as the target sliced up (or down) are combined using Bayesianor Dempster Shafer evidence theory.

Thus it can be seen that the process can be used to identify object ofinterest, such as vehicles, missile launchers. Tests results provided inFIGS. 9A. 9B, 9C, and 9D, have confirmed that the process can identify agreat many objects, with great accuracy. Note that certain results thathave been encircled, and identified by numerals 82A, 82B, 82C and 82Dindicate very low probabilities of error, in the 10 percent range.

While the invention has been described with reference to a particularembodiment, it should be understood that the embodiment is merelyillustrative as there are numerous variations and modifications, whichmay be made by those skilled in the art. Thus, the invention is to beconstrued as being limited only by the spirit and scope of the appendedclaims.

INDUSTRIAL APPLICABILITY

The invention has applicability to the surveillance systems industry.

1. A process for identifying an unknown object, the process comprisingthe steps: compiling data on selected features of a plurality ofsegments of a plurality of known objects; illuminating the unknownobject by means of a laser radar system; dividing the unknown objectinto a plurality segments corresponding to each of the segments of theknown objects; sequentially measuring selected features of each of theplurality of segments of the unknown object; and comparing thesequentially measuring selected features of each of the plurality ofsegments of the unknown object to the selected features on the pluralityof segments of the plurality of known objects.
 2. The process as setforth in claim 1 wherein in the step of compiling data on selectedfeatures on a plurality of segments of a plurality of known objects theplurality of segments of the plurality of known objects are bottom andtop segments.
 3. The process as set forth in claim 1 wherein in the stepof compiling data on selected features on a plurality of segments of aplurality of known objects the plurality of segments of the plurality ofknown objects are bottom, middle and top segments.
 4. The process as setforth in claim 3 wherein the in the step of compiling data on selectedfeatures on a plurality of segments of a plurality of known objectsincludes the step of making piecewise pixel-pair invariant measurementsof each of the plurality of segments of the known objects.
 5. Theprocess as set forth in claim 1, or 2, or 3, or 4, wherein the step ofsequentially measuring selected features of each of the plurality ofsegments of the unknown object includes the steps of comparing thepiecewise pixel-pair invariant measurements of each of the plurality ofsegments of the unknown object until a match is found.
 6. The process asset forth in claim 5 wherein the pixel-pair invariant measurementsinclude the distance between first and second pixels of the pixel-pairs,the angle between the normals, {circumflex over (N)}_(P) and {circumflexover (N)}_(Q), to the to the surface area about the first and secondpixels, the normalized distance between the first and second pairs along{circumflex over (N)}_(P)+{circumflex over (N)}_(Q), and the normalizeddistance between the first and second pairs along {circumflex over(N)}_(P)*{circumflex over (N)}_(Q).
 7. The process as set forth in claim6 wherein the pixel-pair invariants are selected from those havingangles between normals to their respective surfaces of between 80 and100 degrees.
 8. The process as set forth in claim 7 wherein the step ofdividing the unknown object into a plurality segments corresponding toeach of the segments of the known objects includes the step ofdetermining the slope of the ground the object is residing on.
 9. Theprocess as set forth in claim 8 wherein the step of making piecewisepixel-pair invariant measurements of each of the plurality of segmentsof the unknown object is conducted with the source of the laser radar atan angle of between 30 and 60 degrees to the slop of the unknown object.